SUMMARY 


\ 


A  random  walk  or^the  integers  is  considered  with  transition  , 
;  robatilities  pjj*  for  the  transition  k  to  k  ♦  1  and  1  -  p£  for  the 
transition  k  to  k  -  1,  Distributions  and  luonients  are  found  for 
the  length  cf  tine  required  to  travel  between  any  two  given  states, 
bone  Uniting  results,  applicable  to  more  general  results,  are 


1.  Introduction. 


We  consider  Markov  chains  with  dsmaasrable  states,  designated  by 
0,  1,  2,  •••,  and  with  transition  probabilities  independent  of  time. 
Letting  Xq,  x^,  be  the  states  after  0,  1,  •••,  steps,  we  define 

(1.1)  P(n)(l,  J)  ■  PUn-Jl^-l),  n  -  0,  1,  •••, 

where  F(  A  j  B)  stands  for  the  conditional  probability  of  A,  given  B.  V.e 
assume  that  for  each  i  and  J  there  is  an  integer  n  -  n(i,  J)  such  that 

(1.2)  P(n)(i,  J)  >  0  for  n  -  n(i,  j)  . 


Let  be  the  first-passage  time  flron  i  to  jj  is  the  smallest 
positive  integer  n  “  (ch  that  xr  ■  j,  if  xQ  »  i.  If  there  is  no  n  such 
that  xr  •  J,  then  -  ao.  If  J  »  i  we  speak  of  the  recurrence  time 
for  the  state  i. 

We  shall  usually  make  the  assumption 


(1.3)  E(NU)  <  oo. 

If  (1.2)  holds,  then  (1.3)  (which  is  true  for  all  i  if  it  is  true 
for  ary  i)  implies  the  existence  of  a  set  of  stationary  probabili¬ 

ties  >  0  aatisfying 

(1.4)  *  -  li»  i  £  ?<r)(l,  J)  . 

J  n-*oo  n  r-0 
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Ste  Feller  [4],  Chapter  15,  for  the  relevant  theory. 

Let  ©^  be  defined  as  the  probability  that  the  state,  initially  supposed 
to  be  i,  takes  on  the  value  J  at  least  once  before  returning  to  i.  The 
quantities  ©^  turn  out  to  be  very  useful. 

In  Section  2  we  derive  some  identities  to  be  used  in  the  sequel.  In 
Section  3  *«  consider  the  distribution  of  the  recurrence  time  under 
the  Assum;  tions  (1.2)  ani  (1.3),  for  'rare*  states  —  i.e.,  states  for  which 
is  srr.all,  Cince  (assuming  that  there  are  infinitely  mapy  states),  no 
mat  ter  how  the  st  ates  are  numbered,  we  must  have  — ►  0  as  k  — oo ,  we  can 

speak  of  the  distribution  of  for  large  k.  It  is  shown  that 

U-5)  r("k«kONkk  >  u)  '  «ko(e‘U  *  fk(u>)  *  u  "■  °* 


where  £^(u) — >0  as  k — ►  co  for  each  fixed  u  >  0. 

In  Section  4  we  give  explicit  expressions  for  the  n^,  ©^,  and  for  mean 
recurrence  and  first  passage  times,  in  the  case  where  the  Markov  chain  is  a 
random  walk;  that  is,  ■  P^,  ^  ■  1  -  p^.  The  method  depends  on 

the  representation  of  a  random  walk  as  a  Bro..nian  particle  moving  among 
suit  ibly  selected  points,  Section  5  give?  a  more  precise  form  of  Lemma  3  for 
random  walks  an  i  ,v  method  for  jetting  moments  of  first-passage  times  in  r:rdoc 
walks.  Sectior  6  gives  a  rather  curious  corres;>c;  dence  between  random  walks 


ind  trees. 

The  :uthor  in  correspondence  with  wofeesors  Chung  m.1  Feller  while 
this  ,  :.per  was  being  written,  and  both  of  them  furnished  altemrP  ive  ;  I'oofs 
of  some  of  the  results  of  Cection  4.  fore  of  the  identities  in  Cection  2  'ire 
closely  related  to  recent  ml  current  work  of  Chung  [2],  who,  in  ;  articular, 
has  a  result  involving  three  states  of  whir  Lerma  1  of  section  _  is  a  s;ec:al 

-’ase.  AccorJin;  to  ChunL ,  Lemma  a;  pears  in  i  .^ork  ly  :  aul  levy  [7  ]  which  the 

author  has  not  yet  seen.  Dr.  Chung  was  courteous  enough  to  delay  publication 
of  his  paper  until  the  present  paper  was  ready. 


2.  Son#  Identities. 


Leona  1 


1 


Under  the  assumptions  (1.2)  and  (1.3),  we  have 


u.d 


E(NtJ  .  ■  4) 


1  /  J. 


To  prove  (2*1),  suppose  the  Initial  stats  is  i,  and  1st  ^il^* 

be  the  tine  intervals  between  successive  recurrences  to  i.  Let  the  (random) 
integer  R  designate  the  first  cycle  from  i  back  to  i  during  which  the  state 
J  is  visited  at  least  once.  Then  is  a  sample  value  of 

Nij  *  V  Since  L(Nj^)  ■  l/ii^  and  E(R)  ■  1/0^,  we  have,  making  use  of  a 
slight  modification  of  a  theorem  of  Wald,  [9],  p«g*  52, 


(2.2) 


-  E(NU)E(R) 


» 


which  proves  (2.1). 

Since  the  left  side  of  (2.1)  is  unchanged  if  i  and  J  are  interchanged, 
the  right  side  must  be  also.  This  leads  to  the  identity 


(2.3)  "j^i  “  ®ij  /®ji  * 


U 


Formula  (<i.3)  is  useful  in  the  treatment  of  randon  walks,  since  the  6 
are  easy  to  find  and  tnus  the  caa  be  obtained. 

The  identity  (2.3)  holds  even  if  E(N^)  ■  ao  ,  provided  the  chain  is 
recurrent  —  i.e.,  P(N^  <  oo)  -  1,  and  provided  (3,. 2)  holds.  In  this  case 
the  quantities  are  Doeblln's  ratios  (see  [3])» 

Z.p(r)(k,  J) 

fl./n.  "  11m  -  -  -  —  . 

^  n-»ao  *L  / 


Z  f(r)u>  i) 

r-0 


i  aee  the  last  paragraph  of  the  introduction, 
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i 


I 


The  proof  In  this  case  follows  from  results  of  Chun&  In  [2]  and  will  not 
be  given  here. 

As  an  example  of  the  use  of  (2.3) ,  we  have 

Lenma  2.^  Let  Tn( i )  be  the  number  of  times  the  state  la  i  during  the 
first  n  steps  from  an  arbitrary  starting  point.  Then  1  f  (1.2)  holds  and 
if  the  ch-iin  is  recurrent,  we  have 

U.A)  lln  Y  (J)/T  (1)  -*./«,  ,  1  /  J  , 

n-^oo  J 

with  probability  L,  (Paul  Levy.) 

To  prove  (2.4),  which  is  trivial  if  (1.3)  holns,  we  suppose  for 
sim.licity  that  x^  -  i;  this  does  not  essentially  affect  the  argument. 

From  the  definition  of  0^  it  foHows  that  the  probability  that  the  state 
takes  on  the  value  J  exactly  r  times  between  successive  visits  to  i  is  iven 
by  1  -  0.  .  if  r  »  0  and  8.  .(1  -  0-M)r"’^04,  if  r  >  0.  The  expected  number  of 
visits  to  J  between  visits  to  i  is  thus  0^  /  From  the  strong  law  of 

large  numbers  it  follows  that  if  n^,  n^#  *  *  * ,  are  the  times  at  which  xn  -  i, 
we  have 


(..5)  lim  Y  (j)  /Y  (i)  -  6.  /8  .  . 

k-voo  k  \ 

Similar  reas wr.ing  shows  that  the  sane  limit  is  ipproached  if  n  passes  through 
the  sequence  of  values  (n^)  for  which  x  •  J.  But  since  the  sequences  (n^) 
and  (n/)  are  the  only  values  of  n  for  which  the  ratio  Y  (J)  /  Y  (i)  can  change, 

fv  II  l  i 

the  limit  must  exist,  and  use  of  (2.3)  gives  (-.u). 

T - 

"ee  the  )  .  t  goragrap:.  of  the  introduction. 


3.  Distribution  of  for  large  k. 

We  assume  throughout  this  section  that  (1.2)  and  (1.3)  hold  and  that 
there  are  Infinitely  many  states j  otherwise  the  Markov  chain  is  arbitrary. 

We  first  consider  the  distribution  of  Nq^  .a  k — >oo,  where  0  is  an 
arbitrary  fixed  state. 


Lcana  3.  J-in  P(v®nwNn.  >  u) 
k-»a>  u  UK  UK 


u  >  0. 


Let  u  denote  an  arbitrary  fixed  positive  number  and  denote  by  [z]  the 
largest  integer  <  z.  Suppose  the  initial  state  is-  0  and  let  be  the 
number  of  steps  in  the  first  recurrences  to  0.  Let  A^  be  the  event 

that  k  is  not.  visited  during  the  first  ^u/Qq^]  recurrences  to  0,  and  B^(fc-)» 
t  >  0,  the  event 

and  let  be  the  event 

N0k  >  *  ?)[u/80k]  • 

Then  we  have,  letting  5  denote  the  complement  of  B, 


(3.0) 


p(ck}  ^  P(V  V  “  "  P(V  V 

_  «  Ju/90k"J  _  -u  ... 


Now  p(Ak)  -  a  -  eok) 


e”  as  k— >oo,  and  I  (1.) — >  0,  by  the 


law  of  large  numbers,  both  these  limits  holding  uniformly  on  any  finite  u- 
interval  bounded  away  from  0.  An  aasy  consequence  of  (T.0)  is  that 


1J^f  ‘  (*090kN0k  >  “>  5  •  • 


u  >  0 


A  sirailar  argument  shows  lin  sup  ^  •”  ,  and  Leona  3  follows. 
A  consequence  of  Lemma  3  is  that 


(3.1) 


11«  inf  E(«08okN  )  >  1  . 
k  —►oo 


On  the  other  hand,  from  (*-.l)  we  have 


««0 


u^nOdOkNOk^  4  E^0®0kNk0^  "  1 


From  (;.l),  (3.k),  and  (*.3),  we  have,  recalling  from  (*..3)  that 

\«kC 


C  lim  E(«08okNok)  -  11m  E(y>kONOk>  ’  1 

k  -►ao  :  -♦oo 


(3.4) 


llm  MNk0)/E(N0k) 

k  ~*QD 


We  now  prove 


Theorem  1 .  For  each  u  >  0, 


<3-'>  P(\9kONkk  >  »)  ■  9ko'*'“  *  f-k' 


where  fc,  (u)  — >  0  as  k  — ►  oo . 


The  symbols  in  0.5)  are  defined  in  the  introduction.  The  proof  will 
bring  out  the  intuitive  meaning  of  (?.5).  Cf  course,  any  fixed  state  rather 
than  0  could  be  used. 


Proof,  Let  T^  be  the  expected  value  of  N^,  given  that  the  path  from  k 
back  to  k  is  never  in  0  and  let  be  the  expected  value  of  N^q  given  that 
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th«  path  from  k  to  0  is  never  in  k  after  the  initial  position.  Now  consider 
a  sample  first-passage  from  k  to  0,  There  will  first  be  r  returns  to  k, 
r  •  0,  1,  •••,  before  0  is  visited,  and  finally  a  passage  from  k  to  0  during 
which  k  is  not  revisited.  Since  r  has  the  frequency  function  0^(1  *  9k0) 
we  have 


(3.6) 


EOvo)  -  Tv  T  <\0u  -  ek0)r  ♦  uk 
r  *0 


Now,  supposing  again  that  the  initial  state  is  k,  let  0^  be  a  random 
variable  which  is  1  if  the  state  recurs  to  k  without  being  in  0,  and  0 
otherwise.  Then  for  any  u  >  0  we  have 


(3-7)  p( VWkk  >«)•(»-  VHWkk  >  “I : Dk  ’  » 

*  Sko'^nk®kONkO  4  \®kOH0k  >  u|Dk  ’  °>  • 


Now  E(Nk0l  'Jk  •  0)  -  Uk,  and  from  (3.6)  and  (3.6),  Ufc  <  EtN^)  -  o(^|— )  , 
k— ►  oo.  Therefore,  under  the  hypothesis  ■  0,  the  random  variable 
\®k0^k0  conver8®8  in  distribution  to  0.  Thus,  the  second  term  on  the 
right  side  of  (3.7)  is,  using  Lemma  3,  9j^e"u(l  ♦  £k(u)),  where  fk(u) — ►O 
as  k  — >  oo , 

Since  for  any  nonnegative  random  variable  z  we  have  P(z  >  u)  <  E(z)/u, 
u  >  0,  it  follows  that  the  first  terra  on  the  right  side  of  (3.7)  is  bounded 
by  (using  (3.6)) 


(3-8)  (^r“)  \®koE<NkkiDk  -  •  ^[E'\0)-Uk]  i 


¥  \9kOE<V  • 


iiince  from  (3.4)  we  have  \®k  qL(N^q)  — >  0  ao  k  — >  co  ,  Theorem  1  now 
follows . 

As  we  slull  see  in  the  next  section,  the  limit  inferior  of  _  may 
be  0  as  k — ►  ao .  In  thi3  case  Theorem  1  is  nther  empty  of  content* 


4.  Rand on  walks. 


In  the  remainder  of  this  paper  we  coraider  random  walks  on  the  integers 
with  transition  probabi :  ities  pp  for  r — ►  r  ♦  1  ?nd  q  •  1  -  pf  for  r  — ►  r  -  1. 
oome  of  the  results  run  obviously  be  extended  to  the  case  where  the  transition 
r  — *•  r  has  a  positive  probability. 

During  the  remainder  of  this  paper  we  shall  assuae  that  one  of  the  two 
following  conditions  holds: 


Condition  A:  0  «•  p  <  1,  r  -  1,  2,  •**. 

Condition  there  is  an  integer  u )  >  1  such  that 

0  v.  pr  .  lt  r  -  1,  o)-  1;  -  0. 


Some  of  the  results  holding  under  condition  a  .dll  obviously  be  true  wbetner 
or  not  pw  ■  C.  ''hen  condition  ^  applies,  it  is  tacitly  assumed  that  all 

states  mentioned  have  indices  <  £J. 

Wc  make  the  following  definitions.  Let 


(4.1) 


l  » 

r 


ao 

2  ■  L.  <  oo 


r-l 

IT  (q./pJii 

j-i  J  J 

r 

T  i.. 

j-i  - 

(applicable  when 


r  >  1  ; 

r  .>  0  ; 

lit, ion  A  holds). 


Theorem  2a,  Suppose  condition  A  holds  ;ji i  let  the  initial  state  be 
k  >  0,  The  prot  ability  that  the  state  never  reaches  0  ijs  7.../Z, 

If  Z  ■  oo  ,  z^/Z  is  taken  to  be  0, 
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Ineorem  2b.  Su;  pose  condition  B(oJ)  holds  and  let  the  initial  state 
bek.  0  <  k  <  fa).  The  probability  that  the  state  reaches  fa/  before  it 
reaches  0  is  . 

Proof  of  Theorem  2a.  Let  the  points  0  -  zQ,  z^,  z^,  be  marked  off 
on  t  line  and  suppose  th?t  a  one-dimensional  Brownian  motion  (Wiener 
process)  takes  place  on  the  lane.  As  the  Brownian  particle  moves  about 
let  it  always  bear  as  a  label  the  subscript  of  that  one  of  the  z^  which 
it  has  most  recently  visitei.  The  label  then  executes  a  random  walk. 

Using  the  well-known  fact  that  a  Brownian  particle  initially  at  a  point 
0  between  points  A  and  B  has  probability  Ob/AB  of  reaching  A  before  B, 
we  can  verify  that  the  random  w  ilk  execute  :  by  the  label  has  the  3ame 
transition  probabilities  pr,  as  the  original  random  walk. 

Suppose  first  that  Z  -  lim  z  •  oo .  Since  the  graph  of  the  position 

r-^oo  r 

of  the  particle  is  continuous,  every  path  from  z,  to  zn  corresponds  to  a 
definite  finite  se’uence  of  labels  —  i.e,f  a  definite  walk  from  k  to  0. 
Conversely,  every  walk  from  k  to  0  corresponds  to  a  family  of  Brownian 
paths  from  z^  to  families  corresponding  to  distinct  walks  being 
muturally  exclusive.  The  probability  of  a  iven  walk  is  the  same  as  the 
probability  of  the  corresponding  Brownian  family.  Since  almost  all 
Brownian  paths  frotn  z^  reach  Zq,  almost  all  random  walks  from  k  reach  0, 

r*  t  n 

Si  •  *-•>>  • 

lx  2  <  oo ,  walks  from  k  which  reach  0  correspond  to  Brownian  paths 
from  z^  which  reach  0  before  reaching  Z,  the  probability  of  these  being 
1  -  zk/Z,  Q.F.C. 

The  proof  of  Theorem  2b  is  essentially  the  sane. 
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'Jsing  Theorem  2,  c  m  t  ow  evaluate  the  juantities  0^  for  rruom 
walks.  (We  recall  that  0^  Is  the  probability  that  the  state,  initially 
at  i,  reaches  j  at  letst  once  before  returning  to  i.)  In  fact,  if  i  «  J 

we  have 


(4.2) 


e 


ij 


1  ♦ 


*!♦! 

*!♦! 


qi*lqi *2 
Pi*lPi*2 


li*l 


•q , 


-l 


i  "  it 


Pi-l**,pJ-l 


and  a  similar  expression  hoi  is  for  1  'j.  If  i  ■  J  -  1,  the  expression 
on  the  right  of  (4.«t)  is  taken  as  p^. 

Using  (4.2)  ani  (<J.3),  we  can  evaluate  the  stationary  p-obab: lit  ie 
for  random  walks  with  finite  mean  recurrence  times  (or  t.he  L'oeblir.  ratios 
for  recurrent  walks  with  in.finite  mean  times).  For  cx«nple,  putting  i  -  0, 
j  >  0,  we  have  from  (*-.3),  (4,<0»  '<ni  th"  trvilo  lie  of  (4,2)  for  the  Ci.e 

i  <  1, 


(4.3) 


*0®0j/gJ0 


Vo 


M 

1  ♦  —  ♦ 


Vl 


pl' *  *P '-i 


1  ♦  —  ♦ 

ra 

,  a 


qi  i 

■  k 

p,  •  •  *p  .  , 

1  2-1/ 


rh  ,,,|h 


-1 


O-r.-i  o 


Vo 

Yj* 


J  -  0, 


where  Lj  is  defined  by  (4,1).  A  similar  expression  holds  for  J  <  0,  In 
the  case  of  finite  ne  ui  recurrence  times  the  condition  3"  Wj  *  1  completes 
the  detenninat.  ion  in  (4.3)# 


A 3  »n  e>:.’»r.|  le  of  the  use  of  Theorem  2  v/e  have  the  following 


Corollary.  According  A  <>r  13(h))  holds  we  rave 


CD 


-««.»>  •  £  £ 


)  -  £  -i- 

j  "I 


For  we  can  plice  a  reflecting  barrier  it  C,  giving  p 


0 


Then 


•V  ■ '  «W  - 1  ■  ^  -  *•  tIf  EW  '-‘a.  ‘  u) 


‘'inc*-  for  ever/  i  <  J  we  have 


N«. 


i,i^i  Vi,i<  *Vij 


it  is  clear  how  from  Theorem  2  any  desired  mean  recurrence  or  first-passage 
time  car.  re  found. 

As  ci'ntionei  in  the  introduction,  rofesaors  Chung  an'  Feller  have  -iven 
alternate  a;  ro  icnes  ( correaporidence) . 

It  is  interert  ing  no*  t.  cor.r.i  *tr  ran  ion  w-ilr.s  of  the  transient  type, 
e  sh  1 11  s'-'O  mat  the  re  ■  r^  nonrecurrent  rar.  :on  v.alxs  where  th°  con  iltjonal 
me  m  recurrence  time .  riven  th  gt  recurrence  tal.es  pi  ace .  i_s  finite  V.e  pick 
.ij  a  tyf  ic.il  first-passage  r  Jidoo  vari  lie. 


T  eorem  ? .  Suppose  con i ition  A  holds .  The  cor  ci  t lonal  expectation  of 
N^o*  f.lvcn  th  it  N’iq  <  oo  ,  is 


( »U) 


“(niq!  nio 


00  )  «  1 


VI 


0%) 


CD 

r-1 


(l  -  zr/.:)(i  -  zp4l/z) 

— 

r^l 


oo 

whe re  z_  and  L_  are  ie fined  in  (A,l),  uni  Z  -  2  . 

*  r  ~  — — — i  * 

r*l 


This  can  also  be  seen  from  results  of  Chun#  [2]. 
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Before  proving  Thtoren  3*  it  ia  interesting  to  consider  a  special 
example.  Suppose  for  simplicity  that  •  1  and  that  the  quantities 
have  the  fora 

pr  m\  4  %  *  K?1)  »  d>1»  r  -*  cd  , 

where  c  is  independent  of  r.  From  Theorems  2  and  3  we  see  that  if  c  <  -  £ 
the  mean  recurrence  times  are  finite;  if  -  —  <  c  —  .  the  mean  recurrence 
times  are  infinite  but  the  states  arc  recurrent;  if  c  >  t  ,  the  recurrence 

H 

probabilities  are  less  than  1.  In  the ‘latter  case  E(N^q|N^q  <  oo)  is 

3  1  ** 

finite  or  infinite  accordir^;  as  c  >  £  or  £  ^  c  <  £  .  intuitive 

significance  of  this  last  fact  is  that  if  c  is  Large  (i.e.,  c  >  ^) ,  then 

a  path  from  1  is  very  unlikely  to  get  hack  to  G  unless  it  :oe?  ro  quickly. 

To  prove  Theorem  3  it  is  convenient  to  return  to  the  Bro^ni  n  motion 

scheme.  I  f  Z  -  oo  ,  then  Theorem  3  reduces  to  the  corollary  to  Theorem  2 

oo  ^  oo  p  ♦  q  oo  .  oo  i 

(note  the  identity  ^  — -m —  •  ^  -r— : — -  ■  £  .  ■  ■  ♦  r—  j  so  we  may 

r»l  i*  r  r-1  ^r  r  r-1  r*l  r*l  r 

as  well  take  the  case  Z  <  'jo;  Z  i3  of  course  >  1.  V.e  note  that  if  a 
Brownian  particle  is  initially  at  z ^  ■  1,  the  conditional  probability 
density  for  the  maximum  displacement  y  attained  by  the  particle  before  it 
arrives  it  0,  given  that  y  <  Z,  is  l/[(i  -  i/Z)y^J. 

Consider  a  Brownian  path  starting  at  *  1,  and  let  M (y)  he  the 
expected  number  of  label  changes  before  reaching  y  >  z^#  p*  ven  that  y  is 
reached  before  Zq  •  0.  (We  shall  always  suppose  that  y  one  of  the 

points  z  .)  It  is  intuitively  obvious  (we  omit  the  format.  x>f,  which  In 

r 

not  difficult)  that  the  corJitional  expectation  of  the  number  of  label 
changes  in  oinr  from  y  back  to  ?.q,  :iven  that  y  is  the  maximum  J isplacer.ent 
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is  M(y)  ♦  1,  the  added  1  being  for  the  change  which  occurs  when  zQ  is 
reached.  Therefore  the  total  conlitional  expectation  in  a  path  from  z^ 
to  Zq,  given  that  y  is  the  maximum  displacement,  is  1  ♦  2M(y) .  Thus 

(4.4)  S(N10lN10  *  oo)  •  1  ♦  !  _  {  ^2^  • 


Clearly  M(y)  »  0  for  z^  <  y  <  To  evaluate  K (y)  for  y  >  Z2, 

suppose  that  z^  is  the  rightmost  of  thrf  points  z^,  z^,  **•  lying  to  the 
left  of  y,  and  consider  a  Brownian  path  from  z^  to  y  which  reaches  y 
before  z^.  Let  Rj#l<J<k-l,  be  the  total  number  of  label  changes 
which  this  path  would  undergo  if  only  z4  and  z5^,  and  no  other  points  of 
the  set  z^,  z  .,  •••,  were  present,  and  if  the  label  were  initially 
taken  to  he  J.  The  total  number  of  changes  is  then  Rp._^ 

(it  helps  to- draw  a  diagram);  hence 


k-1 

(4.5)  K(y)  -  X^R.lD) 

J-l  J 

where  D  is  the  event  that  the  path,  starting  from  z^,  reaches  y  before  0. 

Now  ^(Rj|D)  can  be  found  by  elementary  combinatorial  means  since,  translated 
back  in  terms  of  random  walks,  a  random  walk  with  only  four  states  is 
involved.  In  fact 


(4.6) 


E(Rj|D) 


l/y 


*J-1  '  ZJ 


From  (4.4),  (4.5),  and  (4. to)  we  have 


r-207 
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(4.7)  E(N1q|N10  <  oo)  -  1  ♦ 


-  .  N  *k^l  k-l/z.  ♦  z  ,  , -2z,z,  ./A, 


/  0  \  N-l 

tAtO  inn  z 

Vi  -  iA‘/N_>00  j.i 


Npl  (1-  «/»!!♦!) C1- 


ZJ*1  ”  ZJ 


«  1  ♦ 


2  \  ®  (l-a/Z)(l-aul/Z) 


W*)5 


*>1  '  Zi 


The  { assa.^e  to  limit  under  the  summation  sign  is  justified  because  each  tern 
in  the  sun  is  positive  and  increases  with  N.  This  gives  Theorem  3. 

If  condition  B(u>)  holds,  then  we  have  the  following  analogue  of 
Theoran  3»  whose  proof  is  entirely  similar: 

The  expected  value  of  N.q,  yiiven  that  0  is  reached  before  CO,  is,  for  U)  >  2, 


(4.P)  E(N1q|0  before  CJ)  -  1  ♦  ^ 


2  \^2  a  -  »A.)d  -  vAP 

17TJA  l _ , 


It  is  now  simple  to  find  the  coniition.il  expected  value  of  N^q  for  any  k  >  0, 
given  that  0  is  reached  before  U)  >  k.  For  we  may  write  ♦  •••  ♦ 

where  the  terns  in  the  sua  are  independent,  hence 

k-1 

E(Nk0l°  before  (ji  >  •  Z)«»k.J(k.>1|0  befor.  V)  . 


Now  E(Nk_j  k_4_jJO  before  U))  •  E(Nk_j  k_._^|k  -  j  -  1  before  oi),  because 
of  the  Markovian  nature  of  the  process.  From  (;..8)f  by  a  translation  of 


indices,  we  have 


(4.9)  E(N,  .  Jj  -  1  before  6d )  -  1 

%j  9  J"’*' 


kZW~ZJ-ln^“  V  rZ 


■  L 
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and,  as  j<olnted  out  before.  Theorem  2  is  In  this  case  rather  devoid  of 
content.  On  the  other  hand,  if  the  limit  superior  of  is  <  1  as 

k— ►od  ,  it  is  clear  that  the  limit  inferior  of  is  positive. 
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5 .  Generating  funct  ions  1  or  r  inhora-walk  variables. 

It  is  customary  to  treat  random  walk  ;  roblrms  by  nears  of  lir.eir  difference 
©quitlor.3.  In  this  section  we  use  a  sinple  nonlinear  recurrence  relation 
whicn  seems  well  adapted  .0  certain  random  walk  problems  and  which,  as  far 
as  the  uthor  knows,  has  not  been  used  before. 

V.’e  ^ive  explicit  expressions  for  the  second  n  nents  of  typical  first- 
pnssr-i^e  variables.  We  shall  indie  ite  briefly  how  these  c  tn  be  used  to  pive 
a  more  precise  fora,  for  random  walks,  of  the  exponential  1  iu.it irv;  results 
of  Jection  3;  hemma  3  holds  for  some  walks  with  infinite  mean  recurrence  times. 


Theorem  4.  Assume  either  condition  A  or  B OtaJ)  of  lection  4  holds .  Let 
be  the  ,,ener  atlng  function  for  the  random  variable  ^(1  ♦ 

00  4  /I  4  N.  ,  ,  \ 


Then  h^(s)  satisfies  the  recursion  formula 


(5.1) 


h^(s)  *  *^k^k-l^b/  ^  * 


w  *  1  >  •  •  • 

K.  »  1,  , 


vh^re 

Tk(u)  -  Pk  /  (1  -  • 

Theorem  4  holds  even  if  h^(l;  <•  1;  i.c.,  even  if  *1^  has  a  positive 
probability  of  beinp  Infinite. 

The  relation  (3.1)  is  analojouc  to  certain  formulae  which  appear  in  the 
theory  of  ir  inching  stochastic  processes,  As  wo  shall  see  in  Section  6,  the 
resemblance  is  more  than  ou;  erfici.il , 
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To  prove  (5.1)  we  first  consider  the  generating  function  of  k^, 
which  we  designate  ij  fk(s).  In  a  passage  from  k  to  k  ♦  1  there  will  first 
be  r  occurrences  of  the  following  event:  the  state  goes  directly  from  k  to 
k  -  1  and  then  continues  changing  until  its  first  arrival  back  at  k.  After 
r  such  occurrences,  the  state  goes  directly  to  k  ♦  1,  For  a  fixed  r  ■  0,  1, 

2,  * ’ * ,  the  generating  function  for  the  number  of  steps  required  is  s(sfk  ^(sj)r. 
Since  the  probability  for  a  given  r  is  p.  (q^)1*  we  have 


(:.2) 


fk(s) 


OB 


H  pk^qk^rs^sfk-l^9^r 

r-0 


Since  sfk(s)  ■  h^s^),  (5.1)  fallows  from  (5.2).  There  is  no  difficulty 
about  infinite  values  of  N’k  k^,  provided  we  adopt  the  convention,  in  the 
argument  just  given,  that  even  when  is  infinite  the  state  does  finally 

go  from  k  to  k  ♦  1  after  Infinitely  many  steps. 

In  the  special  case  of  a  reflecting  barrier  at  0f  h^s)  -  s  and  (5.1) 
gives  a  neans  of  obtaining  h^s)  for  all  k. 

It  is  clear  that  the  moments  of  ^(1  ♦  N,^  k^)  satisfy  recursion  relations 
which  can  be  obtained  by  differentiating  (5.1)  and  putting  s  »  1.  For  ex  unple, 
setting 

"k  '  2<X  *  Vk.l> 

we  have 


(5.3)  .\(1)  -  F(Mk  «  ®)  -  Tk(h,K-1(D)  . 

In  the  recurrent  c^se  loth  3i1es  of  (5.3)  are  equal  to  1.  Differentiation 
gives  the  relations  (which  we  write  down  only  for  the  recurrent  case) 


(5.4) 


Edg  -  1  ♦  J»  E(Vl>  * 


(5.5) 


-  V  ■  t*  E<\-i>E‘V  •  r  «£i  *  Vi>  • 


For  the  reflecting  barrier  case  (where  •  1)  we  have  Mq  •  j(l  ♦  N^)  •  1, 
and  (5.4)  give* 


(5  A) 


E(iy  •$*(!♦  \,k*)-i 


^  W-i 


k  PkPk-l 


♦  •  •  •  ♦ 


qkqk-r,,qi 

Vk-i"V 


which  agrees,  after  a  transformation,  with  Theorms  3.  Similarly,  we  can  use 
(5.5)  to  obtain  the  variance  of  in  the  reflecting  barrier  case. 


obtaining 


2  V*qk  JL  /Pi  *  *  *P-\  ? 

(5.7)  Varianc.  (N^)  .  4(  V  b,)  .  8  (^-b,) 


if  there  is  a  reflecting  barrier  at  0,  where 


,  .  1  ♦  ♦  ^E=j  ♦  ...  qr'Wq;i= 

r  pr  Vr-]— pl 


At  this  point  it  is  convenient  to  write  down  an  expression  derived  from 
(5.7)  by  a  change  of  indicesi  if  p^  -  0  ( reflecting  barrier  at  k),  then 

k  ,  \  /  k  ,  \  k-1  f  ,  k  ,  \ ;  k 


(5.8) 


Variance 


where 


-  1,  L  -  — - r  >  1. 

”  is  ■  «  *n  " 


r  pl***pr-l 
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Froo  (5.8)  it  follows  that  in  ai>y  recurrent  random  walk  where  condition  A 


of  Section  4  holds  we  have 


(5.9)  Variance  (N^Q) 


^(lf)(!f)*8f.fr(!f)(i  i-y 

'r«l  r''r»2  Lr '  r-l[  ' j-r  y 


We  now  re-examine  the  arguments  leading  to  Leaoa  3  of  Section  3, 


*or  simplicity  wo  assume  throughout  the  rest  of  this  section  that 


Condition  C:  Pq  ■  1 


holds . 

In  deriving  Lemma  3*  the  assumption  of  a  finite  mean  recurrence  tine 
enabled  us  to  say  that  [u/®Qk]  cycles  from  0  to  0  correspond,  with  high 
probability,  to  almost  exactly  — jj— -  steps  when  k  is  large.  In  the  case  of 

Vok 

random  walks,  however,  it  is  clear  that  the  distribution  of  does  not 
depend  on  the  value  of  the  p^  for  r  >  k.  Therefore,  lot  us  suppose  that 
the  original  random  walk  is  altered  by  placing  a  reflecting  barrier  at  k  >  0; 
since  condition  C  holds,  this  will  make  the  recurrence  times  finite. 

The  modified  random  walk  will  have  a  new  set  of  stationary  probabilities 
*r(k),  r  -  0,  1,  •••,  k.  The  question  is  then  whether  [u/®QkJ  cycles  from  0 
to  0  in  the  modified  random  walk  correspond  almost  exactly  to 
steps.  We  know  that  the  number  of  steps  in  a  cycle  of  the  modified  walk,  having 
a  finite  mean  value,  satisfies  the  weak  law  of  large  numbers,  but  is  [u/®Qk] 
cycles  a  #large  enough"  number  to  make  the  sample  mean  almost  equal  to  the 
true  mean?  Clearly  it  is  sufficient  to  have 

lio  (nQ(k))2  ®Qk  Variance  (N10(k))  -  0 
k-*oo 


(5.10) 


wl..«e  H^(k)  it  th#  random  variable  N^p  in  the  modified  random  walk.  To 
(5*10)  to  ary  particular  case,  we  can  refer  to  (4.2)  and  (5 .8). 

At  an  example  of  a  case  where  the  mean  recurrence  times  are  infinite 
but  (5.10)  etill  holds,  suppose  Pr  *  J  •  r  ■  1,  2,  •••«  Prom  Theorems  2 
and  3  this  corresponds  to  a  recurrent  random  walk  with  infinite  mean  recurrence 
times.  Then 


WVi 

Wpr-1 


~  c,r, 


r— *oo, 


where  is  independent  of  r,  and  an  easy  calculation  shows  that  (5.10) 


Although  we  shall  net  enter  into. details  here,  tilt  results  of  this  s set ilp 
can  be  used  to  give  error  terms  for  Theorem  1  or  Lemma  3  in  the  case  of  random 
walks  with  finitely  many  states.  In  particular,  soma  of  the  results  of  [l] 
for  the  time-continuous  Ehrenfest  modal  can  be  obtained  for  the  usual  Ehrenfest 
model  with  a  discrete  time  parameter.  The  argument  ie  simple  but  tedious, 
depending  on  the  use  of  Chebyshev's  inequality. 


6.  Walks  and  trMs,^ 

Random  walks  and  branching  processes  are  both  objects  of  considerable 
interest  in  probability  theory.  We  may  consider  a  random  walk  as  a 
probability  measure  on  sequences  of  steps  ~  that  is,  on  "walks,1  as  defined 
below.  A  branching  process  is  a  probability  measure  on  •trees,"  as  defined 
below.  The  purpose  of  the  present  section  is  to  show  that  walks  and  trees 
are  abstractly  identical  objects  and  to  give  probabilistic  consequences  of 
this  correspondence.  The  Identity  referred  to  is  non-probabilietic  and  is 
quite  distinct  from  the  fact  that  a  branching  process,  as  a  Markov  process, 
may  be  considered  in  a  certain  sense  to  be  a  random  walk,  and  also  distinct  from 
th<*  fact  that  each  step  of  the  random  walk,  having  two  possible  directions, 
represents  a  twofold  branching. 

By  a  ■walk"  we  shall  mean  ary  finite  sequence  of  integers  n^,  n^,  nr 

satisfying  the  following  conditions: 

(6.1)  nQ  -  nr  -  0, 

nj  >  0,  1  <  j  <  r  -  1, 

|nj  -  njal  -1,  J  •  0,  1,  *•*,  r  -  1  . 


Notice  that  our  walks  begin  at  0  and  terminate  as  soon  as  0  is  reached,  and 
we  consider  for  the  time  being  only  those  which  do  come  back  to  0. 

By  a  "tree"  we  shall  mean  a  finite  set  of  objects  having  the  relationshipe 
of  a  (male)  family  tree  descended  from  a  single  ancestor.  A  typical  element 
of  the  tree  can  be  designated  by  a  symbol  of  the  form 


I.  J.  Good  has  pointed  out  the  similarity  between  certain  formulae  in  branching 
processes  and  random  walks,  [5).  Mr.  Good  has  also  informed  me  by  letter  that 
D.  G.  Kendall  has  recently  shown  a  relationship  between  branching  processes  and 
the  theory  of  queues,  and  Good  himself  has  shown  a  connection  between  the  theory 
of  queues  and  the  gambler's  ruin  problem. 


•  •  • 


meaning  the  mp-th  son  of  the  •••  of  the  m^-nd  son  of  the  m^-st  eon  of  the 
original  ancestor.  For  our  purpose  the  two  tree* 


(6,2) 


are  distinct  objects,  since  we  keep  track  of  the  ‘order  of  birth*  of  the 
sons  of  a  given  father. 

To  exhibit  the  correspondence,  we  lay  down  the  general  principle  that 
a  step  to  the  right  in  the  walk,  say,  froa  k  to  k  ♦  1,  corresponds  to  a  birth 
of  an  object  in  the  k-th  generation.  The  'parent"  of  this  step  is  the  last 
preceding  step  from  k  -  1  to  k.  The  'children*  of  a  given  step  from  k  to 
k  ♦  1  (let  us  call  this  step  S)  are  the  steps  frosi  Y  ♦  1  to  k  ♦  2  which  occur 
after  S,  but  before  any  step  from  k  ♦  1  to  k  succeeding  S.  The  'children" 
are  numbered  in  the  order  of  appearance  ~  the  first  step  is  oldest,  etc. 

A  step  to  the  left,  say,  from  k  ♦  1  to  k,  means  that  the  person  corre¬ 
sponding  to  the  last  preceding  step  frosi  k  to  k  ♦  1  has  died  and  will  have 
no  further  issue. 

Rather  than  jiving  a  tedious  fornal  demonstration  of  the  correspondence, 
we  shall  here  only  illustrate  it.  The  reader  can  easily  convince  himself 
by  working  through  c  few  such  examples.  Consider  then  the  leftmost  of  the 
two  trees  in  (6.2).  This  corresponds  to  the  walk  whose  successive  positions 
are  0,  1,  2,  3*  2,  3,  2,  1,  2,  1,  2,  1,  0,  if  we  adopt  the  convention  that  of  two 
vertices  in  the  same  generation  pictured  in  the  tree,  the  upper  is  the  elder. 

The  correspondence,  step  by  step,  is  as  follows  ((0,  1)  means  a  step 

from  0  to  1,  etc.):  (0,  l)<->a  appears;  (1,  2)*->b  is  bom;  (2,  3)**-*c 
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is  lorn;  (3,  2)4— ►  c  dies;  (2,  3)4— M  is  born;  (3,  2)4— ►d  dies; 

( 2 ,  l)4->fc  ilea;  (1,  2)4— ►e  is  tom;  (2,  1)4— *e  dies;  (1,  2)4— >f  is 
bom;  (2,  1 ) 4-4  f  dies;  (1,  0)4— ►»  dies.  Similarly,  the  other  tree  in 
(6.2)  corresponds  to  the  walk  0,  1,  1,  2,  3»  2,  3,  2,  1,  2,  1,  0. 

Now  3upf)ose  that  probabilities  p^,  ^  ■  1  -  are  prescribed  for  the 
random  walk,  with  0  <  p^  <  1,  k  ■  1,  2,  ••*.  Consider  next  the  following 
branching  process:  an  individual  in  the  (k  -  l)st  generation  has  a 
probability  p^q^  of  hAving  exactly  r  children,  r  -  0,  1,  ••'.  (The 
ancestor  ia  the  0  generation.)  Then  we  have  the  result 

Theorem  6 •  The  probability  that  a  random  particle  starting  at  1  reaches 
0  for  the  first  time  after  2r  -  1  steps  equals  the  probability  that  the  family 
tree  just  defined  becomes  extinct  after  producing  a  total  of  r  individuals, 
r  •  1 ,  2,  •  •  • .  The  total  probability  that  the  particle  returns  to  0  equal  a 
the  probability  that  the  family  tree  is  finite. 

The  proof,  which  we  omit,  follows  from  the  identity  between  walks  and 
trees  previously  demonstrated;  it  can  also  be  shown  by  the  use  of  generating 
functions . 

As  an  example  we  consider  the  classical  case  ■  p.  The  expected 

number  of  children  of  a  3ingie  individual  in  the  family  tree  is 
ao  . 

q  rpr  -  —  .  If  p  <  4,  i.e,,  ,  ^  -  <  1,  the  total  maber  of  Individuals 

1-P  “  2'  *l-p-' 

in  the  tree  is  finite  with  probability  1,  and  the  generating  function  ^(s)  of 
the  total  number  satisfies  the  relation 

«•>  •  . 


whence  we  have  (see  Otter  [8],  or  Hawkins  and  Ulam  [6]) 


1  -  a/1  -  4pq» 
2P 


4i»)  - 


It  can  be  verified  that  4(*)  ie  likewise  the  generating  function  for  the  randcn 
variable  ^(1  ♦  N^).  In  the  case  p  >  ^  ,  4(a)  has  the  nans  form,  but  in  this 
case  ^(1)  ■  q/p;  this  is  the  probability  that  the  family  is  finite  and,  as  is 
well  known,  is  likewise  the  probability  that  N^q  Is  finite. 

If  condition  R(uj)  holds,  the  generating  function  of  the  random  variable 
^(1  ♦  N^q)  can  be  obtained  by  changing  the  numbering  system  of  the  lattice 
points  in  Theorem  4*  We  obtain 


(6.3) 


# 

where  Tr(u)  •  q^/U  -  Pru).  This  formula  is  likewise  the  generating  function 
for  the  total  m*ber  of  individuals  in  generations  0  through  U)  -  1  of  the 
family  tree.  The  generating  function  for  2U  ♦  N10)  in  the  case  where 
condition  A  holds  (and  for  the  total  number  of  individuals  in  the  family  tree) 
is  obtained  by  letting  OJ  go  to  oo  in  (6.3)* 
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